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Abstract:  There  are  many  definitions,  descriptions,  and  usages  of  the  term  “mental  model.”  Frequently, 
the  definition  of  mental  model  is  not  described,  leaving  what  the  author  means  as  an  exercise  for  the  reader. 
I  propose  a  very  explicit  definition  for  a  dynamic  mental  model  and  then  show  how  that  definition  can  be 
applied  in  the  domain  of  meteorological  forecasting.  Specifically,  I  suggest  that  a  dynamic  mental  model  is 
a  mix  of  images  and  propositions,  relies  on  qualitative  and  spatial  relationships,  allows  dynamic,  runnable 
results  to  be  inspected,  and  results  in  an  inference.  Finally,  I  offer  suggestions  on  how  to  improve  the 
usefulness  of  the  term  mental  model. 


Many  meteorologists  and  cognitive  scientists  have 
suggested  that  meteorologists  use  a  “mental  model”  to  predict 
the  weather  (Hoffman,  Coffey,  &  Ford,  2000;  Lowe,  1994; 
Perby,  1989;  Trafton  et  al.,  2000).  However,  the  term  “mental 
model”  has  been  used  by  many  different  researchers  to  mean 
many  different  things.  For  example,  mental  models  can  be 
based  on  visual  images  or  abstract  situations  representing  a 
single  possibility  (Byrne,  2004).  These  mental  models  have 
been  used  to  study  reasoning  and  deduction  (Johnson-Laird  & 
Byrne,  1991).  Norman  uses  the  term  mental  model  to  explore 
how  devices  like  computers,  ATMs,  etc.  work  (Norman,  1983, 
1988).  Gentner  describes  a  mental  model  as  ' 'a  representation 
of  some  domain  or  situation  that  supports  understanding, 
reasoning,  and  prediction"  (Gentner,  2001)  and  has  used 
mental  models  to  explore  how  dynamic  systems  unfold 
(among  other  things).  In  general,  the  scientific  community 
uses  the  term  “mental  model”  to  mean  one  of  several  mental 
representations;  additionally,  it  may  have  one  of  several 
different  properties.  If  meteorologists  use  mental  models  to 
help  them  predict  the  weather,  what  exactly  does  that  mean, 
and  what  is  the  evidence  for  their  using  mental  models? 

Instead  of  performing  an  exhaustive  review  of  how 
different  researchers  have  used  the  term  mental  model,  I  will 
instead  describe  the  definition  that  our  group  has  been  using 
the  last  few  years  and  provide  supporting  evidence  for  each 
aspect  of  that  definition.  In  general,  this  paper’s  description  of 
mental  model  will  come  closest  to  that  used  by  Gentner  and 
Stevens  (1983).  Not  coincidentally,  many  weather  forecasters 
use  a  very  similar  (though  equally  fuzzy)  definition  (e.g., 
Perby,  1989).  Our  definition  of  a  mental  model  will  be: 

•  a  mix  of  images  and  propositions 

•  relying  on  qualitative  and  spatial  relationships 

•  allowing  dynamic,  runnable  results  to  be 
mentally  inspected 

•  resulting  in  an  inference 

•  almost  always  requiring  a  great  deal  of  domain 
knowledge 

Evidence  for  each  of  these  points  will  be  discussed  in  the 
sections  below. 

A  mental  model  is  primarily  a  spatial  representation, 
and  this  spatial  information  is  represented  qualitatively  not 
metrically  or  quantitatively.  It  is  a  mix  of  images  and 


propositions  because,  while  most  researchers  see  the 
importance  of  the  imagistic  component,  it  must  be  able  to 
connect  to  our  propositional  reasoning  mechanisms  (Altmann 
&  Trafton,  2002;  Anderson,  Conrad,  &  Corbett,  1989).  It  also 
must  be  “runnable”  (dynamic)  -  the  process  that  makes  a 
mental  model  unique  from  many  other  representations. 

Finally,  a  mental  model  frequently  requires  a  great  deal  of 
thought  and  energy  to  create  compared  to  other  mental 
operations;  the  output  of  a  completed  mental  model  run  is  an 
inference. 

How  well  does  this  definition  match  to  empirical 
work  in  the  meteorological  forecasting  domain?  Recent  work 
suggests  that  meteorologists  do,  indeed,  form  a  mental  model 
while  making  a  forecast  and  use  that  mental  model  to  predict 
what  the  weather  will  be. 

A  mix  of  images  and  propositions 

Recent  empirical  work  suggests  that  weather 
forecasters  do,  indeed  use  a  mix  of  images  and  propositions  to 
make  a  forecast.  Many  cognitive  psychologists  believe  there 
is  a  strong  propositional  component  to  the  way  that  we  think: 
there  are  multiple  computational  cognitive  models  that  rely 
exclusively  on  propositional  accounts  (Anderson  &  Lebiere, 
1998;  Kieras  &  Meyer,  1997;  Laird,  Newell,  &  Rosenbloom, 
1987;  Newell,  1990).  However,  many  cognitive  psychologists 
equally  believe  that  much  of  cognition  has  a  strong  pictorial  or 
imagistic  component  (Kosslyn,  1980,  1990;  Laeng  & 
Teodorescu,  2002;  Thomas,  1999).  In  the  forecasting  domain, 
there  seems  to  be  a  strong  belief  that  weather  forecasters 
create  a  mental  image  in  their  head  of  what  the  weather  is  and 
what  it  will  be.  It  is  not  clear,  of  course,  whether  forecasters’ 
mental  image  is  simply  epiphenomenal  to  the  actual  processes 
or  whether  it  is  the  imagistic  representation  that  is  doing  the 
“work”  (e.g.,  Pylyshyn,  1973).  The  evidence  from  forecasters 
that  suggests  that  working  meteorologists  have  a  mental  model 
is  typically  based  on  idiosyncratic  introspection  and/or  leading 
questions.  The  work  is  thus  not  methodologically  strong  from 
a  human-factors  point  of  view,  but  it  should  be  noted  that  most 
forecasters  truly  believe  they  keep  a  picture  in  their  head  about 
the  weather. 
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Relying  on  qualitative  and  spatial 
relationships 
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example,  the  Winter  2004  issue  of  AI  Magazine,  which 
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what  is  the  evidence  that  forecasters  use  a  qualitative 
representation  to  help  them  forecast  the  weather?  This  issue  is 
especially  relevant  because  the  visualizations  that  a  forecaster 
examines  shows  quantitative  information  to  be  extracted,  a 
forecaster  frequently  makes  a  quantitative  prediction  about 
what  the  weather  will  be  (e.g.,  the  temperature  will  be  72°), 
and,  meteorology  is  considered  a  “hard”  science  with  a  great 
deal  of  physics  and  mathematical  training  traditionally  needed. 
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The  above  figure  is  a  traditional  meteorological 
forecasting  visualization  -  it  shows  temperature  (color  coded), 
sea-level  pressure  (iso-lines  with  numeric  labels),  wind-speed 
and  wind-direction  (wind-barbs)  overlayed  over  most  of  the 
United  States.  Even  though  this  visualization  displays 
quantitative  data,  several  studies  of  expert  forecasters  have 
shown  that  forecasters  actually  extract  primarily  qualitative 
information!  For  example,  a  forecaster  looking  at  a 
visualization  may  say  something  like  “the  temperature  in 
Canada  is  particularly  cool”  rather  than  saying  “the 
temperature  is  52°. ”  What  is  particularly  interesting  about  this 
example  is  that  the  forecaster  must  then  take  their  qualitative 
representation  (or  mental  model)  and  create  a  numeric  forecast 
from  it.  Again,  forecasters  are  able  to  perform  this  task  quite 
accurately.  The  evidence  for  this  “turning  pictures  into 
numbers”  process  occurs,  we  argue,  because  forecasters  create 
a  mental  model  that  connects  the  visualization,  their  expertise 


and  knowledge  of  various  weather  systems,  and  the  reasoning 
processes  (especially  the  dynamics,  discussed  in  the  next 
section).  The  basic  effect  of  extracting  qualitative  information 
and  generating  quantitative  information  has  been  shown  with 
Navy  forecasts  (Trafton  et  al.,  2000)  and  replicated  with 
Australian  meteorologists  (Kirschenbaum,  Trafton,  &  Kramer, 
in  press). 

Allowing  dynamic  results  to  be  mentally 
inspected 

One  of  the  least  controversial  components  of  a 
mental  model  is  that  it  is  a  dynamic  representation  that  allows 
the  “playing  out”  of  different  hypothetical  situations.  For 
example,  if  a  forecaster  is  unsure  exactly  where  a  front  may 
be,  she  may  mentally  play  out  the  previous  location  of  the 
front  (e.g.,  mentally  animate  the  progression  and  movement 
based  on  its  history)  and  decide  based  on  her  own 
understanding  of  the  current  weather  situation  where  it  should 
be.  After  she  has  determined  the  location  of  the  front,  she 
must  also  determine  how  other  variables  will  be  affected  by 
the  front  -  temperature,  wind  speed  and  direction, 
precipitation,  etc.  This  variable  propogation  seems  to  happen 
dynamically. 

Additionally,  the  mental  dynamics  of  weather 
forecasters  as  they  predict  the  weather  has  been  shown  in  both 
laboratory  studies  (Lowe,  1994,  1999)  and  more  naturalistic 
studies  (Bogacz  &  Trafton,  2002).  Additionally,  other 
researchers  have  shown  the  prevalence  of  dynamic  reasoning 
in  other  domains  including  mechanical  diagram  understanding 
(Hegarty,  1992;  Hegarty  &  Sims,  1994)  and  scientific 
reasoning  (Trickett  &  Trafton,  under  review).  Bogacz  and 
Trafton  (2002)  showed  that  experienced  Naval  forecasters 
mentally  animated  static  meteorological  visualizations. 
Interestingly,  the  forecasters  examined  very  few  dynamic 
visualizations  (e.g.,  satellite  loops)  even  though  they  had 
access  to  animations,  suggesting  that  forecasters  would  rather 
mentally  animate  their  own  weather  models  than  see  an 
explicit  animation. 


Resulting  in  an  inference 

A  forecaster’s  weather  prediction  is,  by  definition,  an 
inference:  no  one  knows  with  complete  certainty  what  the 
weather  will  be  in  the  future.  Computational  weather  models 
make  extremely  precise  quantitative  predictions,  but,  as  shown 
above,  forecasters  do  not  rely  solely  on  the  numeric  results  of 
individual  forecasts,  using  instead  the  qualitative  values  and 
relationships  between  different  variables.  Trafton  and  his 
colleagues  have  shown  that  forecasters  combine  the  results  of 
multiple  sources  of  infonnation  to  make  each  quantitative 
prediction  rather  than  simply  reading  off  a  single  numeric 
answer.  For  example,  a  forecaster  may  examine  several 
different  infonnation  sources  and  extract  the  specific 
temperature  over  a  location:  78°,  74°,  and  82°.  The  forecaster 
may  make  a  prediction  that  the  temperature  will  be  76°.  Note 
that  76°  is  not  a  numerical  average,  though  it  is  within  the 
range  of  values.  This  number  is  an  inference  about  what  the 


weather  will  be.  Our  claim  is  that  the  forecaster  has  built  a 
mental  model,  run  it  through  several  possibilities,  and  created 
the  most  likely  scenario  based  on  his  experience  to  create  a 
prediction  for  what  the  temperature  will  be. 


A  reliance  on  domain  knowledge 

In  most  cases,  a  mental  model  is  created  with  a  great 
deal  of  domain  knowledge:  most  mental  models  are  created  in 
domains  where  a  lot  of  knowledge  is  critical  to  solve 
problems.  In  meteorology  this  is  certainly  the  case: 
meteorologists  need  a  great  deal  of  information  to  create 
accurate  forecasts.  One  of  the  open  questions  about  expertise 
and  meteorology  is  how  much  of  a  mental  model  is  created 
and  used  by  novice  or  journeyman  forecasters. 

Summary  and  Discussion 

This  paper  has  suggested  that  a  mental  model  is  a 
mix  of  images  and  propositions  that  consists  of  both 
qualitative  and  spatial  relationships.  A  mental  model  allows 
dynamic,  runnable  results  to  be  mentally  inspected  and  results 
in  an  inference.  Finally,  a  mental  model  usually  comes  from 
strong  domain  knowledge.  This  definition  has  been  explored 
and  supported  in  the  domain  of  meteorology. 

There  are  several  reasons  why  the  definition  of 
mental  models  is  important  in  the  field  of  human  factors. 

First,  imprecise  definitions  hurt  the  science  of  the  field  -  if 
definitions  are  not  explained,  the  meaning  of  the  term 
becomes  diluted  (as  could  be  argued  is  happening  now  with 
the  term  mental  models).  By  sticking  with  a  precise  definition 
of  the  term  mental  model,  researchers  can  tightly  constrain  the 
representational  and  process  assumptions.  Additionally,  if  a 
particular  study  does  not  fit  the  definition  of  a  mental  model, 
different  representational  assumptions  may  be  needed,  and 
different  processes  may  need  to  be  assumed  in  order  to  fully 
account  for  the  study.  Of  primary  importance  is  the  point  that 
the  term  mental  model  should  not  be  used  to  describe  an 
ambiguous  or  unknown  representation  of  a  complex  task,  or 
that  generally  means  “how  something  works.” 

How  does  a  good  description  of  mental  model 
enlighten  us  about  the  cognitive  processes  in  meteorology? 
Another  way  of  asking  that  question  is  “Why  is  knowing  that 
meteorologists  use  mental  models  important?”  One  possible 
answer  is  that  if  there  is  a  mismatch  between  external 
information  and  the  user’s  mental  model-building  capabilities, 
it  is  going  to  be  much  more  difficult  to  build  a  coherent  and 
robust  mental  model.  For  example,  by  our  definition,  a  mental 
model  is  a  mix  of  images  and  propositions.  Currently, 
meteorologists  are  able  to  get  both  imagistic  and  textual 
(propositional)  data  on  the  weather.  Second,  a  mental  model 
relies  on  qualitative  and  spatial  relationships.  This  part  of  the 
definition  strongly  suggests  that  visualizations  should 
emphasize  the  qualitative  aspect  of  data  display  rather  than  the 
purely  numeric.  The  wind-barb  is  an  excellent  example  of  a 
good  qualitative  glyph:  it  shows  at  a  glance  the  direction  and 
speed  of  wind  at  that  location.  Interestingly,  the  quantitative 
information  can  also  be  extracted  from  a  wind-barb. 


Since  a  mental  model  is  dynamic  and  runnable, 
information  should  be  presented  to  facilitate  that  dynamic 
quality.  Simply  providing  animations  is  probably  not  the  best 
way  to  support  a  mental  model  because  a  mental  model  must 
be  run  inside  the  person’s  head  rather  than  on-screen.  Thus, 
visualizations  should  be  presented  in  time-order  so  that  the 
forecaster  can  mentally  animate  it. 

A  mental  model  also  frequently  results  in  an 
inference.  This  aspect  of  a  mental  model  is  probably  the  least 
understood;  there  are  many  ways  to  facilitate  inferences.  One 
possible  way  to  facilitate  inferencing  within  meteorology  is  to 
facilitate  the  comparison  process:  knowing  how  different 
weather  models  differ  from  or  are  similar  to  each  other  (and 
how  different  they  are  from  recent  satellite  pictures)  may 
facilitate  the  overall  inferencing  process. 

In  summary,  researchers  should  precisely  define  what 
they  mean  by  “mental  model.”  This  paper  has  suggested  one 
possible  definition  and  shown  how  it  maps  to  meteorology. 
This  paper  has  also  described  the  advantages  of  using  a  mental 
model  approach  to  mis-matches  between  external  and  internal 
representations.  Importantly,  any  difficulty  a  person  has  with 
a  task  does  not  automatically  mean  there  is  a  mental  model 
mismatch.  These  mis-matches  (based  on  the  definition)  can 
be  used  to  guide  not  only  future  research  but  product 
development  as  well. 
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